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Abstract-Nuclei spotting has been 
given a paramount importance in 
diagnosing and monitoring many 
medical conditions. It also helps the 
pharmacists to develop and discover 
new formulas of drugs/remedy by 
observing the effects of medicines on 
the patients. Nuclei spotting becomes 
a challenging task due to the natural 
variation in the appearance as well as 
the variation of image capturing 
devices. Besides, variation in the 
lightening conditions also pose extra 
challenges in the process of detection 
and segmentation of nuclei. In the 
current study, we employed a 
modified U-Net (mU-Net), a deep 
learning-based approach, for nuclei 
detection and segmentation. The 
results showed the supremacy of the 
proposed method. Intersection over 
Union (IOU) of 0.78 was achieved on 
BBBC038v1 dataset. 


Index Terms-Deep learning, digital 
pathology, nuclei detection, U-Net 


Pakistan 


lL. Introduction 


The current study claimed that 
people from all over the world get 
infected with various types of 
diseases including cancer, chronical 
diseases, brain and heart issues, 


diabetes and Alzheimer’s. The 
drugs for Alzheimer’s and 
Parkinson’s diseases are too 


expensive and takes plenty of time 
to be at one's disposal in the market. 


Therefore, many companies 
working on the research and 
development of drugs for 


Parkinson’s and Alzheimer’s 
diseases have stopped to develop 
new medicines. Recent studies 
indicated that development of a new 
medicine, from target identification 
through approval for marketing, 
takes over 12 years and often much 
longer to discover a new drug [1]. 
The assessment process of most 
diseases has become expeditious 
due to the advancement in the field 
of medical imaging. The 
methodologies of automatic nucleus 
classification, nucleus, and 
detection algorithms have been 
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adopted for the fast diagnosis and 
cure of many critical diseases, that 
is, cancer. The starting point is to 
identify cell nuclei. The analysis 
contains 30 trillion cells. These cells 
contain nuclei which are full of 
DNA of human body. The accurate 
identification of the cell function 
helps the researchers to examine 
how cells react together to discover 
different cures and treatments. 
Moreover, the researchers become 
capable to understand the biological 
processes after such an 
identification at the cell level. This 
solution provides with the faster 
treatment of patients and better 
outcomes in terms of medicines 
discovery. Therefore, the field of 
microscopy imaging and 
computational pathology play a 
cardinal role in the diagnosis of 
different diseases. Furthermore, 
these images contain a lot of 
information for CAD. Therefore, 
the throughput/output rate for the 
analysis of images can become very 
high. Currently, the computational- 
pathology has become spearhead 
and renowned in the research of 
medical imaging. Also, it has 
become serviceable for the patients 
as well as for the pathologists. 
Thus, the computational-pathology 
has achieved significant attention 
from the researchers and the clinical 
practice community [2]. As the 
computational techniques give 


efficient outcome and accelerates 
the process of image analysis as 
compared to the traditional and 
manual systems for clinicians and 
researchers. In manual image 
analysis, the computational- 
pathology and the microscopic- 
imaging are very challenging 
procedures to deal with. Therefore, 
it leads to large inter-observer 
variations. Besides, CAD 
characterizes the diseases very 
accurately and minimizes the bias in 
a significant way. Moreover, the 
computational pathology provides 
measurements of pathological 
images which can be reproduced for 
clinical follow-ups. The current 
study aims to personalize treatment 
process and drug’s discovery, which 
is ultimately helpful for patients. 
The computerized approach gives 
fast image analysis as compared to 


the time consuming manual 
assessment, which releases the 
researchers from repeated and 


boring routine efforts. In order to 
handle the huge amount of dataset, 
grid computing and computationally 
scalable algorithms provide high 
throughput/output for analyzing 
pathology images. Whereas, the 
nuclei detection and segmentation is 
the critical prerequisite of CAD and 
it is also considered as a basis of 
automated image analysis. 
Furthermore, it provides analysis for 
calculating size, shape, 
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morphology, and texture of cells. 
However, a robust and accurate 
segmentation of nuclei is very 
difficult. In histopathology and 
microscopy, images contain 
background clutter, noise, blurred- 
region during image acquisition, 
and poor contrast. Moreover, there 
are variations in size, shape, and 
intracellular intensity of nuclei. In 
the similar manner, nuclei are very 
close to each other and often overlap 
with one another. Figure 1, 
pictorially explains the detection 
and segmentation of Nuclei. Many 
researches have been conducted for 
the sake of automatic nucleus 
detection and segmentation, which 
is eventually aimed to improve 
detection accuracy. 


The proposed research focuses 
on the improved nuclei 
segmentation with cluttered 
background, varying acquisition 
devices, and natural variation of 
nucleus structure. Besides, the 
presented work performs 
modification in U-Net deep 
learning based architecture. Also, 
the results based on the 
BBBC038v1 dataset [4] validates 
the primacy/priority of the proposed 
methodology. The proposed 
research significantly improves the 
segmentation accuracy of the state 
of art work. 
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II. Related Work 


The literature of this study 
showed that there are several 
different state-of-the-art methods 
for detecting and localizing the 
nuclei. Lin et al. [5] and 
Bhattacharjee et al. [6] perform very 
well on the detection process, but do 
not control output masks. Lu et. al 
[7] demonstrates the effectiveness 
of combining the object proposals 
and the edge detection together. As 
they utilize the unsupervised 
training session, so we are unsure if 
it would have been competitive to 
implement in this research or not. In 
this case, others have focused on 
detecting tumors and have been 
classifying them, while this research 
does not focus on classification. 


The feature-based 
representation utilized in Khoshdeli 
et. al [8] shows promise as a 
preprocessing method, but the most 
difficult problem in constructing 
good masks for the data in this 
research is separating the 
overlapping nuclei. Therefore, we 
are unsure if this feature-based 
representation would help with this 
issue or not. Since, it is mainly a tool 
for accentuating nuclei from the 
background. In Xie et al. [9] focus 
on the cell detection by outputting a 
cell centroid but not a mask. Thus, 
they would require adaption in 
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procedure, if they were to be used 
for different data. 


However, the researchers 
discovered that this problem is an 
instance segmentation problem 


which is a complex issue and one of 
the most challenging computer 
vision tasks, which can further 


perform instance segmentation by 
(b) Nuclei Detection 


(a) Original Image 


detecting the objects and predicting 
the pixel-level instances on objects. 
There are several state-of-the-art 
methods for instance segmentation 
in medical imaging. Few methods 
focused on the overlapping of cells. 
Dynamic priors and fuse deep 
CNNs are utilized for the above 
mentioned problem [5, 8]. 


(c) Nuclei Segmentation 


Fig. 1. Nuclei detection and segmentation process [3] 


They have annotated their own 
dataset which they used for 
evaluation procedure, so there are 
no comparable results. In Chen et al. 
[10] design a deep contour-aware 
network. Their method and Tareef 
et al. [11] method achieves state-of- 
the-art results on their respective 
datasets. The difference between 
these and He et al. [12] is that Mask 
R- CNN does not focus on medical 
imaging and is not as heavily 
engineered. Mask R-CNN is a 
single model as well as easier to 
adapt and implement. However, it is 
the state-of-the-art for more general 
instance segmentation. In addition 
to it, it is more interesting to 


experiment with adapting a general 
instance segmentation model. It is 
especially conducted to solve the 
specific problem of instance 
segmentation of nuclei in medical 
imaging. The results from various 
methods are difficult to compare as 
they have not been evaluated on the 
same datasets. Table 1, summarizes 
the different state of the art 
techniques from the literature 
review of the current study. 


According to the Literature 
review of the current study, the state 
of the arts are not focused on the 
overlapping of the cells, occluded 
nuclei. Moreover, limited attention 
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was given to the device independent 
solutions. The dataset used in this 
research was too resourceful, even 
having highly cluttered 
backgrounds and overlapping 
nuclei. The proposed methodology 


IlI.Methodology 


In this section, the researchers 
would represent the details on 
nuclei detection after using the 
modified UNet model, which is 


‘nificantly i eae end-to-end fully convolutional 
signilicantly HAPIOYE e detection network (FCN). 
and segmentation accuracy. 
Table I 
Summary of Literature Review Techniques 
Paper Method Dataset Result Evaluation 
Deep Learning Method 
Delineates Multiple 
Nuclear Phenotypes in H & ENet Private 0.6904 F-Score 
E Stained Histology 
Section[8] 
ScanNet:A Fast & Dense ScanNet Camelyonl6 0.8533 FROC 
Scanning Framework foe [23] 
Metastatic Breast cancer 
Detection from Whole Side 
Images[5] 
: Mean 
[ods.ai] topcoders Kaggle BBBC038v1 
; Unet 0.6316 Average 
2018 Data Science [13] [4] Precision 
jacobkic Kapele 2018 Data UNetRCNN BBBC038v1 0.6147 Mean 
: [4] Average 
Science Bowl[[13] Precisi 
recision 
. Mean 
Deep Retina Kaggle 2018 MRCNN BBBC038v1 0.6140 Average 
Data Science Bowl [13] [4] Precis; 
recision 
They first present the dataset and A. Dataset 


the architecture of the currently 
proposed Methodology. Thus, they 
discuss each phase of the proposed 
methodology. 
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As discussed earlier, different 
datasets have been used for nuclei 
detection. In this research, they 
employed BBBC038v1 dataset for 
training and testing purpose. 
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Whereas, it was available from 
the Broad Bioimage Benchmark 
Collection and was the standard 
dataset for the Kaggle competition. 
All the images in the dataset had a 
height between 256 and 524 pixels 
and a width of between 161 and 696 
pixels. The number of nuclei varied 
greatly; from only a couple to 
several hundred. The images had 
been captured under the different 
conditions. Furthermore, they 
varied in color, lighting, and the 
cells varied in types of cell. The 
dataset consisted of a training set of 
670 images. On average there were 
29461/670 = 44 nuclei per training 
image. Eventually, there was a 
variety of background and 
foreground images in the dataset. 
There can be 


1. Black foreground and white 
background 

2. Purple background and (purple, 
white, yellow) foreground. 


3. White foreground and black 
background 
B. Proposed Method 


The currently proposed 
methodology is based on U-Net. 
The height and width of the images 
are normalized to fed the proposed 
model. The proposed model is a 
modified version of U-Net. ELU 
activation function is used in their 
proposed solution. Where they 
employed BBBC038v1 dataset for 
conducting the training program and 
to fulfill the validation purpose. 
They also used 603 images as the 
training set and validation was 
performed on 67 images which was 
10% of the total dataset. 65 images 
were used to evaluate the 
performance of our proposed 
solution. The basic architecture of 
the proposed methodology was 
given in Figure 2. The details of 
each step of the proposed solution 
was explained in the subsequent 
section. 


Data Preprocessing 


Nuclei 
Detection 


| Test Data - 


Preprocessing 


Fig. 2. Architectural diagram of proposed methodology 
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1. Data Preprocessing 


In data pre-processing, data is 
modified or transformed into the 
useful format. Data can also exhibit 
many issues while data generation, 
therefore there is a need to pre- 
process it before using it. As 
explained earlier, the images in the 
dataset had the different height and 
width. In the pre-processing step, 
the size of all images is set to 
256x256, so that we can pass them 
to the model for training. Suppose, 
the image I has the size w x h, and 
the resultant image is of size 
w' x h' , then for the each pixel in 
the input image, the new 
transformed coordinated would be 


(1) 


The resized images are then 
normalized 


ran Iy’) 
A(x, y") = OEE (2) 
After the pre-processing of 


images, the images are fed to the 
proposed model for the nuclei 
detection. 


2. Overview of U-Net Aechitecture 


The currently proposed 
methodology is modified U-Net. U- 
Net is very popular for medical 
image segmentation. It is a fully 
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convolutional network (FCN). 
There is a concept of skipping 
connection between different stages 
of U-Net, which gives superiority to 
its design [14]. 


The input-output mapping of 
linear networks can be represented 


Out, = Wp-1Wp-2 WX (3) 


Where Out, is the output and X 
is the input of the network. 
Moreover, W; is the weight matrix 
between layer j and j +1 and j 
spans from 1 to n — 1. For network 


with skip connections, this 
formulation is modified as 

Out,” = (Wn-1 + I)(Wn-2 + 

D) ..... (W1 + DX (4) 


Here Out," is the output of skip 
connection network, having n 
layers. For non-linear networks, the 
input-output mapping can be 
formulated as 


Outn f (Wn-1f (Wn-2f C... nE 
5 


Out” = f((Wn-1 + Df (Wn-2 + 
Df (.....£(W, + DX)))) (6) 


f is the activation function. 
However, it applies some changes to 
deal with the trade-off between 
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localization and the use of context. 
This trade-off begins at the large 
patches, where there is a need of 
producing more pooling layers and 
reduce the accuracy of the 
localization. On the contrary, small 
patches can only perceive a small 
context of the information. There 
are two paths in U-Net model: 
analysis and synthesis. In the 
analysis path, U-Net follows CNN 
structure. The synthesis path is also 
called as the expansion phase. It 
contains an up-sampling layer and a 
deconvolution layer. Moreover, the 
important feature of U-Net is the 
shortcut connections between the 
layers. These connections exist 
between the layers which have an 
equal resolution. The deconvolution 
layers get high-resolution features 
from these connections. This 
architecture resembles the English 
alphabet ‘U’, therefore, it is called 
U-Net. U-Net is composed of the 
following three parts: The 
contraction section, the bottleneck 
section, and the expansion section. 
Furthermore, there are many 
contraction blocks in the contraction 
section. Each contraction block of 
this section takes an input and 
employs two convolution layers of 
size 3X3. After applying 
convolutions it applies a 2X2 max 


pooling. After each block, the 
number of the feature maps 
becomes double. Therefore, 


architecture is the learning of too 
complicated structures. Hence, the 
last layer negotiates between the 
contraction layer and the expansion 
layer. In this layer, two 3X3 CNN 
layers are used and followed by 2X2 
up convolution layer. Since, 
convolution is used without 
padding, the size of the output is less 
than the input size. Therefore, the 
strategy of overlapping is used 
instead of downsizing before 
network and up-sampling after the 
network. Thus, the image is 
predicted piece by piece in this 
technique. 


3. Modified U-Net as Proposed 
Solution 


Since unpadded convolution is 
used, the output size is smaller than 
the input size. Instead of downsizing 
before network and up-sampling 
after network, overlap tile strategy 
is used. Thereby, the whole image is 
predicted part by part. 


In the original paper, the size of 
the input image is 572x572x3, 
however, they would use the input 
image of size 256x256x3. It largely 
decreases the feature size and 
computational cost. Hence, the size 
at various locations may differ from 
that in the original paper but the 
core components remain the same. 


In this proposed solution, the 
researchers modified the U-Net 
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model to improve the accuracy of 
nuclei detection. For this purpose, 
they made some changes in Base U- 
Net architecture. The U-Net 
architecture uses RELU as the 
activation function. In the proposed 
solution, RELU is replaced with 
ELU [15]. Relu gives input in the 
range of 0 to a max value and it also 
gives the positive values against all 
the negative values. Converting all 
negative values directly into zero is 
a drawback and it is called "dying 
ReLU". In such a situation a ReLU 
neuron becomes “dead”, it is stuck 
in case of the negative values and 
always gives 0 output. Because its 
slope in case of a negative range is 
also 0. ReLU cannot recover a 
neuron once its value becomes 0, 
the neuron becomes useless and it 
cannot play any role in 
distinguishing the input. In the 
result, a large portion of the 
network would not take any part in 
the training. 


Anyhow, the activation function 
named as an Exponential Linear 
Unit or its widely known name ELU 
was used. Researchers revealed that 
the function converges cost to zero 
faster and gives very accurate 
results. Therefore, ELU was 
adopted in our proposed solution to 
improve the performance of the 
model. Mathematically ELU is 
defined as 
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e~—1 ;forx<0 
fwd =| x; forx 20 


(7) 
For back propagation, the 
derivative of positive values will be 
one and for negative values that 
derivative will be e*. These 
properties lead to faster training and 
better accuracies. Also, it does not 
suffer from dying neuron problem 
or exploding/vanishing gradients. 


Moreover, the researchers 
introduce a drop-out layer between 
two convolutional layers of every 
block to avoid overfitting. Large 
neural network trained on relatively 
small datasets can over fit the 
training data. This has led to the 
effects of the model learning 
focusing on the statistical noise in 
the training data. Furthermore, it 
results into the poor performance 
when the model is evaluated on new 
data, that is, a test dataset. Hence, 
the generalization decreases due to 
the overfitting process as the 
generalization error increases. The 
architecture of proposed 
methodology (modified U-Net) is 
shown in Figure 3. As the output 
Softmax function is applied on each 
pixel of the resultant image to 
achieve multi-class segmentation 
problem. Cross-entropy function is 
used for loss calculation and 
updating weights. Softmax function 
and cross entropy is defined as 
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a(Z)i = Seed (8) 


-Xf tilog(o(Z);) (9) 


Where Z is an input vector and o 
is the softmax output. 


LOoSScR = 


We used Adam optimizer with 
the learning rate to 0.001 for 
training. Adam optimizer is used to 
accelerate the gradient descent 
algorithm. It includes exponentially 
weighted average of the gradients, 


Input layer 


Fig. 3. Modified U-Net architecture. 
IV.Results & Discussion 


This section presents the details 
of the experiments which were 
carried out to evaluate our proposed 
methodology. The researchers 
conducted different experiments for 
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resulting in faster convergence. 
Mathematically, 


Wei. = Wr-% & (10) 


ôLoss 
e = Bia + (1-8) [=E] 
(11) 
Where, 
č, = aggregate of gradient at time t 


x= learning rate 


B = Moving average parameter 


—> Conv 3x3, elu 
= dropout 


l Max pool 2 x 2 


T Up-conv, 
— Conv1ixi1 
copy and crop 


the evaluation of proposed solution. 
They used Google Colab with 12GB 
memory and 48 GB Disk space to 
perform our experiments. They 
employed BBBC038v1 dataset and 
computed very promising results. 
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The details of dataset division and 
computed results are explained in 
subsequent section. 


The dataset contains 735 images 
in total, we used 603 images for 
training and 67 images for 
validation purpose. They used the 
above mentioned experimental 
settings and performed three 
different experiments. IOU is the 
evaluation metrics, which is mostly 
employed in object detection 
problem to measure the accuracy of 
model. The IOU is computed 
between the ground truth bounding 
box of original images and predicted 
bounding box. The IOU between 
predicted pixels (A) of model and 
ground truth pixels (B) can be 
computed as 


PNQ 


IOU(P,Q) = F59 


(12) 

IOU is the most commonly used 
metric for object detection and 
segmentation. IOU is a vital metric 
to track the tasks of human 
annotation. Therefore, it is widely 
used in medical imaging problems. 
Other evaluation metrics like MAP 
(Mean Average Precision) does not 
show that how closely it has been 
predicted the output and how far it 
matches with the actual outcome. 


Department of Information Systems 


Volume 2 Issue 1, Spring 2022 


IOUs tell precisely that how much 
the prediction matches/deviates 
from the ground truth. 


They also used ELU as the 
activation function. Moreover, a 
dropout has also been introduced 
between the every two 
convolutional layers so that we can 
avoid overfitting. 


The model is trained on different 
epochs where we realized that at 
19th epoch, the training loss is 
minimum and value of IOU is 
maximum. Therefore, we stopped 
the training process and saved the 
model. The training and validation 
loss of the model is shown in Fig. 4. 
Also, the training and validation 
accuracy of the model is shown in 
Fig 5. 


In order to evaluate the trained 
model, the researchers used 65 
images as a test set, where they fed 
the test images to model and model 
computed IOU between ground 
truth information of original image 
and predicted labels. They also 
computed 0.78 IOU value over the 
test set which is greater than the 
previously reported results. The 
sample image of nuclei detection is 
shown in Fig. 6. 
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Fig. 4. Training and validation accuracy of proposed methodology. 
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Fig. 5. Training and validation loss of proposed methodology. 
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Fig. 6. Pictorial representation of nuclei detection. 


Moreover, for the verificationof second experiment RELU is used as 
our proposed solution we performed the activation function. They did 
some more experiments. In the not normalize the input and no 
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dropout is used in this presented training and validation loss of the 
setup. The model is trained on model is shown in Fig. 7. Also, the 
different epochs. They computed training and validation accuracy of 
0.63 IOU value over test set. The the model is shown in Fig. 


MIOU 


—e—Training = Validation 


Number of Epochs 


Fig. 7. Training and validation accuracy using RELU activation function 
without dropout layer. 


Loss 


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 


— Training = Validation 
Number of Epochs 


Fig. 8. Training and validation loss using RELU activation function 
without dropout layer. 


In the third experiment we used with dropout layer. The model is 
RELU as the activation function. trained on different epochs. They 
They performed the experiment computed 0.58 IOU value over the 
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test set. The training and validation accuracy of the model is shown in 
loss of the model is shown in Fig. 9. Fig. 10. 
Also, the training and validation 


MIOU 


1 2 3 4 5 6 7 8 9 10 11 12 13 14 1s 16 17 18 19 


—e—Training 9—8—Validation 


Number of Epochs 


Fig. 9. Training and validation accuracy using RELU as activation 
function with dropout layer. 
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Fig. 10. Training and validation loss using RELU as activation function 
with dropout layer. 


In the fourth experiment we with normalization of data. The 
used RELU as the activation model is trained on different epochs. 
function. They performed the They computed 0.67 IOU value 
experiment with dropout layer and over the test set. The training and 
UMT Artificial Intelligence Review 
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validation loss of the model is 
shown in Fig. 11. Also, the training 
and validation accuracy of the 
model is shown in Fig. 12. Hence, it 


is clear from the above-conducted 
experiments, that our proposed 
solution gives better results with 
0.78 IOU value. 


MIOU 
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Fig. 11. Training and validation accuracy using RELU as activation 
function with dropout layer and lambda function. 
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Fig. 12.Training and validation Loss using RELU as activation function 
with dropout layer and lambda function. 


As the researchers are inspired 
from Kaggle Nuclei Detection 
Competition, so they are going to 
compare the proposed solution 
results with the top scorer of that 
competition. Table 2 shows the 
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comparison of the proposed scheme 
with state of the art. Below is given 
the comparison of that Table which 
shows the proposed methodology 
which improved the accuracy of 
nuclei detection. 
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Table II 
Comparison of Results of Proposed Methodology 
Paper Method Dataset Result Evaluation 
Deep Learning Method 
Delineates Multiple 
Nuclear Phenotypes in H ENet Private 
& E Stained Histology 0.6904 F-Score 
Section [5] 
Scan Net: A Fast & 
Dense Scanning 
Framework foe Camelyon16 
Metastatic Breast cancer Seanet [16] UBS?° BROS 
Detection from Whole 
Side Images [18] 
[ods.ai] Top coders Mean 
Kaggle 2018 Data Science UNet BBBC038v1 0.6316 Average 
[13] Precision 
: Mean 
Jacobkie Kaggle 2018 = UNetRCNN  BBBC038vl_-0.6147.—Ss Average 
Data Science Bowl|[[13] Precisi 
recision 
; Mean 
Deep Retina Kaggle 2018  ARECNN:  BBBCOSYI Olio  -Avéragé 
Data Science Bowl [13] Precisi 
recision 
f Mean 
Proposed Methodology gee BBBC038v1 0.78 Average 
S Precision 


V. Conclusion 


The current study focused on the 
detection and the localization of 
nuclei in medical images by using a 
modified and advanced U-Net 
model. In this study, researchers 
employed the modified version of 
U-NET in order to detect the nuclei 
in medical images. Whereas, 
publicly available standard 
benchmark BBBC038V1 is used for 
model training and testing in this 


research. Dataset contains strenuous 
images with overlapping nuclei, 
cluttered backgrounds, varying 
lightening conditions and variety of 
acquisition devices, and 
configurations. The proposed 
research works well on_ these 
challenging test cases. The 
researchers performed different 
experiments for the evaluation of 
the proposed solutions. They also 
observed the presented model 
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improved accuracy (IOU value) 
24% better than the non-modified 
model. IOU score of 0.78 over test 
set is reported by the presented 
research. From now on, the 
combination of different models can 
be used to improve the detection of 
nuclei. Moreover, U-Net model can 
also be modified to increase the 
detection and localization rate, that 
is, Ladder-Net, which also can be 
used for such complex detection 
tasks. Hence, the different advanced 
techniques can be used to solve this 
complex detection problem. 
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